Papers with learning algorithms
Variational Inference and Deep Generative Models (P18-5)
Copied to clipboard
| Challenge: | Unsupervised and semi-supervised learning has been addressed scarcely in NLP . this tutorial provides an introduction to variational inference followed by an example-driven discussion of how to use variational methods for training DGMs. |
| Approach: | This tutorial provides an introduction to variational inference followed by an example-driven discussion of how to use variational methods for training DGMs. |
| Outcome: | This tutorial provides an introduction to variational inference followed by an example-driven discussion of how to use variational methods for training DGMs. |
Meta Learning and Its Applications to Natural Language Processing (2021.acl-tutorials)
Copied to clipboard
| Challenge: | Meta-learning is a new technique that aims to learn better learning algorithms, including better parameter initialization, optimization strategy, network architecture, distance metrics, and beyond. |
| Approach: | This tutorial introduces Meta-learning approaches and the theory behind them, and then reviews the works of applying this technology to NLP problems. |
| Outcome: | This tutorial will introduce Meta-learning approaches and the theory behind them, and then review the works of applying this technology to NLP problems. |
Bias and Fairness in Natural Language Processing (D19-2)
Copied to clipboard
| Challenge: | a tutorial will review the history of bias and fairness studies in machine learning and language processing . |
| Approach: | This tutorial reviews the history of bias and fairness studies in machine learning and language processing . it presents recent community effort to quantify and mitigat bias in natural language processing models . |
| Outcome: | This tutorial reviews the history of bias and fairness studies in machine learning and language processing . it aims to quantify and mitigate bias in natural language processing models for a wide spectrum of tasks . |
Unlearn Dataset Bias in Natural Language Inference by Fitting the Residual (D19-61)
Copied to clipboard
| Challenge: | Statistical natural language inference models are susceptible to learning dataset bias. |
| Approach: | They propose a debiasing algorithm that debiases models that use only known dataset biases . they use two benchmark datasets to train three high-performing NLI models . |
| Outcome: | The proposed learning objective improves model performance on challenge datasets while maintaining reasonable performance on original datasets. |
Evolutionary Strategies at Scale lead to Catastrophic Forgetting (2026.acl-short)
Copied to clipboard
| Challenge: | ES has been shown to improve performance on specific tasks, but it is accompanied by significant forgetting of prior abilities. |
| Approach: | They propose to use Evolutionary Strategies to train gradient-free algorithms to improve performance. |
| Outcome: | The proposed algorithm achieves performance numbers closer to GRPO for math and reasoning tasks, but forgets prior abilities. |
DomiKnowS: A Library for Integration of Symbolic Domain Knowledge in Deep Learning (2021.emnlp-demo)
Copied to clipboard
| Challenge: | Current deep learning architectures are data-hungry with issues mainly in generalizability and explainability. |
| Approach: | They propose a library for the integration of domain knowledge in deep learning architectures . structure of data is expressed symbolically via graph declarations and constraints can be added to deep models . |
| Outcome: | The proposed framework simplifies programming for integration of domain knowledge in deep learning architectures while separating the knowledge representation from learning algorithms. |
Meta-Learning for Domain Generalization in Semantic Parsing (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to parsing use standard supervised learning, but little attention has been given to domain generalization. |
| Approach: | They propose a meta-learning framework which targets zero-shot domain generalization for semantic parsing. |
| Outcome: | The proposed framework significantly boosts parser performance on English and Chinese spider datasets. |
An Analysis under a Unified Formulation of Learning Algorithms with Output Constraints (2024.acl-srw)
Copied to clipboard
| Challenge: | Existing work on NN models with output constraints has not been able to categorize them in a unified manner. |
| Approach: | They propose new algorithms to integrate the information of main task and constraint injection . they use the H-score as a metric for considering main task metric and constrain infringement simultaneously . |
| Outcome: | The proposed algorithms integrate the information of main task and constraint injection, inspired by continual-learning algorithms. |
Meta Learning for Natural Language Processing: A Survey (2022.naacl-main)
Copied to clipboard
| Challenge: | Meta-learning is an emerging field in machine learning, but there is no systematic survey of these approaches in NLP. |
| Approach: | They propose to introduce meta-learning and the common approaches and summarize their work and review their work in the NLP community. |
| Outcome: | The proposed methods improve performance in many NLP tasks but are limited to domains, languages, countries, or styles. |
Complicate Then Simplify: A Novel Way to Explore Pre-trained Models for Text Classification (2022.coling-1)
Copied to clipboard
| Challenge: | Existing frameworks for text classification employing pre-trained models are constrained by the difficulty of the task. |
| Approach: | They propose a framework which implements a two-stage training strategy to fully exploit the knowledge in pre-trained models. |
| Outcome: | The proposed framework outperforms state-of-the-art classification models on six text classification corpora. |
Uncovering the Limits of Text-based Emotion Detection (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Identifying emotions from text is crucial for a variety of downstream tasks. |
| Approach: | They consider the two largest now-available corpora for emotion classification: GoEmotions and Vent. |
| Outcome: | The proposed models outperform the two largest corpora for emotion classification: GoEmotions and Vent. |
Policy Shaping and Generalized Update Equations for Semantic Parsing from Denotations (D18-1)
Copied to clipboard
| Challenge: | Existing learning approaches for parsing from denotations (SpFD) do not provide access to correct representations, so there are two steps for every training example. |
| Approach: | They propose a framework for parsing from denotations that generalizes three different learning algorithms. |
| Outcome: | The proposed framework outperforms previous work by 5.0% absolute on exact match accuracy on a question answering dataset. |
Constructing Distributions of Variation in Referring Expression Type from Corpora for Model Evaluation (2022.lrec-1)
Copied to clipboard
| Challenge: | referencing is a non-deterministic task, but the algorithms for RE generation are evaluated against corpora of written texts which only include one RE per reference. |
| Approach: | They propose a method for exploring variation in human RE choice on the basis of longitudinal corpora. |
| Outcome: | The proposed method shows agreement between the evaluations against human judgements and parallel evaluations. |
Understanding Learning Dynamics Of Language Models with SVCCA (N19-1)
Copied to clipboard
| Challenge: | a new study shows that neural models implicitly encode linguistic features . but no research shows how these encodings arise as the models are trained . |
| Approach: | They propose a method that compares learning across time and across models using annotated data. |
| Outcome: | The proposed method compares learned representations across time and across models without evaluation on annotated data. |
Learning Transferable Feature Representations Using Neural Networks (P19-1)
Copied to clipboard
| Challenge: | Traditional domain adaptation algorithms learn common representations which suffer from transfer loss when the source specific characteristics detract their ability to represent the target data. |
| Approach: | They propose to segregate source specific representation from the common representation and use it to learn a two-part representation which captures source specific characteristics while the second part captures the truly common representation. |
| Outcome: | The proposed representation outperforms existing learning algorithms on the source learning as well as cross-domain tasks on multiple datasets. |
Preference Consistency Matters: Enhancing Preference Learning in Language Models with Automated Self-Curation of Training Corpora (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods to address inconsistencies in preference learning datasets rely on heuristics to achieve alignment. |
| Approach: | They propose a method that preprocesses annotated datasets by leveraging proxy models trained directly on them to detect and select consistent annotations. |
| Outcome: | The proposed method shows performance improvements of up to 33% across learning algorithms and proxy capabilities. |
FETA: A Benchmark for Few-Sample Task Transfer in Open-Domain Dialogue (2022.emnlp-main)
Copied to clipboard
Alon Albalak, Yi-Lin Tuan, Pegah Jandaghi, Connor Pryor, Luke Yoffe, Deepak Ramachandran, Lise Getoor, Jay Pujara, William Yang Wang
| Challenge: | Prior studies of task transfer in dialogue consider only 2-4 tasks, focus on multitasks. |
| Approach: | They propose a benchmark for FEw-sample TAsk transfer in open-domain dialogue. |
| Outcome: | The proposed benchmark analyzes the transferability between 132 source-target task pairs and provides a baseline for future work. |
Exploring the Learning Capabilities of Language Models using LEVERWORLDS (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing models of stochastic learning involve learning general structure rules and specific properties of the instance. |
| Approach: | They propose a framework that allows the generation of physics-inspired worlds that follow a similar generative process with different distributions and their instances can be expressed in natural language. |
| Outcome: | The proposed framework allows the generation of physics-inspired worlds that follow a similar generative process with different distributions and their instances can be expressed in natural language. |